include: devicetree: Generate defines for map property #87595

soburi · 2025-03-24T22:05:29Z

This change introduces generating definitions corresponding to
*-map property, which was currently discarded.

The *-map data is like a two-dimensional array, so it is difficult to
handle with the existing APIs, so we will also provide new APIs.

Common utilities have been separated into a separate PR.
Please take a look at this first. #84190

I noticed this problem while developing https://github.com/zephyrproject-rtos/gsoc-2022-arduino-core.

Since map is used to associate with the physical layout, there is a need to know the "physical pin names".

Since the mapping process is performed during the generation of devicetree-generated.h, I don't think there are many cases where information about the mapping source is needed. However, there are such cases, however rare, and it is not desirable for the information written in the devicetree to be discarded, so I would like to introduce this.

rruuaanng · 2025-07-17T14:42:54Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

+                raw = raw[4:]
+
+                controller_node = prop.node.dt.phandle2node.get(phandle)
+                if not controller_node:


Try to use is None, it's more readable.

Sorry for my late response.
I fixed it.

rruuaanng · 2025-07-17T14:43:50Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

+                    _err(f"controller node cannot be found from phandle:{phandle}")
+
+                controller: Node = self.edt._node2enode[controller_node]
+                if not controller:


+1

#87595 (comment)

rruuaanng · 2025-07-17T14:52:35Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

+                    # Not enough room for phandle
+                    _err("bad value for " + repr(prop))
+
+                def count_cells_num(node: dtlib_Node, specifier: str) -> int:


It's recommended to move it outside the while loop. Placing the function inside the loop looks odd.

Fixed it as your suggestion.

sonarqubecloud · 2025-08-02T05:36:34Z

Quality Gate passed

Issues
8 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

soburi · 2025-09-10T07:54:36Z

@rruuaanng

Is it possible to proceed with this PR? If there are any issues, please let me know and I will address them.

rruuaanng

Looks good, let's hear the suggestions from the other developers.

rruuaanng · 2025-09-11T05:31:37Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

It would be better to add a test for the maps() function in scripts\dts\python-devicetree\tests\test_edtlib.py.

Thank you for your comment. I’ve added the test you suggested.
This PR has been open for quite some time, so there has been enough time for feedback.
If there are no outstanding issues, could you please approve it?
If anyone has objections, I’m sure they’ll raise them. 😀

mbolivar

Quick first pass review -- I very much like the idea. Can you please update the documentation as well so we can track these new macros? I also want to see the macros.bnf update so I can check that the implementation matches it.

mbolivar · 2025-10-03T16:08:03Z

tests/unit/util/main.c

+	uint8_t a[] = {GET_ARGS_FIRST_N(0, 1, 2, 3)};
+	uint8_t b[] = {GET_ARGS_FIRST_N(1, 1, 2, 3)};
+	uint8_t c[] = {GET_ARGS_FIRST_N(2, 1, 2, 3)};
+	uint8_t d[] = {GET_ARGS_FIRST_N(3, 1, 2, 3)};


What is the intended behavior if N > length(__VA_ARGS__)?

Thank you for your review.

This will result in a compilation error.
Specifically, the N portion is a non-variable-length argument, so it will be detected as an argument mismatch. This allows for early error detection. (delightful!)

/home/crs/zephyrproject/zephyr/tests/unit/util/main.c:568:8: error: macro "Z_GET_ARGS_FIRST_4" requires 5 arguments, but only 3 given 568 | uint8_t e[] = {GET_ARGS_FIRST_N(4, 1, 2, 3)};

dts/bindings/pcie/host/pci-host-ecam-generic.yaml

mbolivar · 2025-10-03T16:12:34Z

scripts/dts/gen_defines.py

            out_dt_define(macro, val)


+def write_maps(node: edtlib.Node) -> None:


It's a requirement to update macros.bnf in the documentation when we add new generated macros. Please add it.

I updated doc/build/dts/macros.bnf.

Gets the specified number of elements from the beginning of the list. This is the symmetircal operation as `GET_ARGS_LESS_N`. Signed-off-by: TOKITA Hiroshi <[email protected]>

Add `GET_ARGS_FIRST_N` tests. Signed-off-by: TOKITA Hiroshi <[email protected]>

mbolivar

@soburi thank you for your update.

The first 4 commits look good to me.

I ran into some issues while reviewing commit number 5, scripts: dts: Add handling for *-map property. Please check these comments. Anything marked "nonblocking" is just an idea that won't block the merge from my side.

dts/bindings/iio/io-channel-nexus.yaml

dts/bindings/interrupt-controller/interrupt-nexus.yaml

mbolivar · 2025-10-04T16:01:31Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

+            if node is None:
+                _err("node is None.")


This should be impossible if the type checking is working properly. The only way node could be None is if its type annotation was dtlib_Node | None. The fact that the type is dtlib_Node already means it cannot beNone. If that is not true, something is wrong with the type checking or the type annotation.

Admittedly, this is redundant if there is type checking.
Removed.

mbolivar · 2025-10-04T16:05:42Z

scripts/dts/python-devicetree/src/devicetree/edtlib.py

        return res

+    @property
+    def maps(self) -> list[ControllerAndData]:


This is adding a new user-facing API to edtlib without documenting it. Please compare this with the way other properties (like gpio_hogs) are written and documented and add the missing documentation.

It should look like this:

@property def maps(self) -> list[ControllerAndData]: "See the class docstring" ...

And then the class docstring for Node should be extended to document the maps property appropriately.

It's hard for me to check if the function is working correctly if its behavior is not documented. It's also necessary to document all the public APIs so users of this python package know how they are supposed to work.

Added description. Please check it.

doc/build/dts/macros.bnf

They must be `array` type to match the existing `interrupt-map-mask` and `interrupt-map-pass-thru`. And also, updating descriptions. Signed-off-by: TOKITA Hiroshi <[email protected]>

Add a base binding for interrupt and io-channel nexus nodes. This makes the implicit definitions explicit. Replace existing individual map definitions with this. This file does not define a specific `compatible` on its own, So you should include this file before using it practically. Signed-off-by: TOKITA Hiroshi <[email protected]>

This change introduces generating definitions corresponding to `*-map` property, which was currently discarded. For `*-map` properties are made able to be treated as a variation of phandle-array, assign sequential cell names for each group of specifiers (child_specifier_0, child_specifier_1, ..., parent_specifier_0, ...). The `*-map` data is like a two-dimensional array, so it is difficult to handle with the existing APIs, so we will also provide new APIs. Signed-off-by: TOKITA Hiroshi <[email protected]>

Add a map property test for `test_edtlib.py`. Signed-off-by: TOKITA Hiroshi <[email protected]>

Add tests for map property related macros. Add `native_sim/native/64` to allowed platforms. Signed-off-by: TOKITA Hiroshi <[email protected]>

sonarqubecloud · 2025-10-05T12:09:52Z

Quality Gate passed

Issues
9 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

mbolivar

@soburi thank you for your update. I think there is enough in here for me to give this a good review, which I will do as soon as I can. (I am traveling this week on a work trip so my availability will be a bit less than it was last week, but I want to prioritize your PR among other DT review requests I have.)

As an overall design question, I'm not sure that I think that ControllerAndData is an appropriate class to be using to represent the maps.

The zephyr phandle-array type is used when we have a list of (phandle, specifier) pairs. But in the case of generic maps, we don't have pairs, we have triples: (child_specifier, specifier_parent_phandle, parent_specifier). And in the case of interrupt maps, we have 5-tuples: (child_unit_address, child_interrupt_specifier, interrupt_parent, parent_unit_address, parent_interrupt_specifier).

I think that neither of these types of information is really a ControllerAndData due to the additional "child -> parent" information.

I am currently thinking it would make more sense to introduce two new types:

NexusMapEntry: for the table entries contained in a <specifier>-map as documented in DTSpec section 2.5.1. This would contain attributes named child_specifier, parent, parent_specifier.
InterruptMapEntry: similar, except for the interrupt-map property table entries in DTSpec 2.4.3.

This would match the same pattern we have done with the Range type used to represent elements of the standard ranges property. Experience has shown from that case that picking type names and designing APIs to match the specification leads to future-proof work.

What do you think about this idea?

doc/build/dts/macros.bnf

mbolivar · 2025-10-05T17:27:24Z

I'm not sure that I think that ControllerAndData

#87595 (review)

cc @dottspina for his perspective as a consumer of this API

soburi · 2025-10-06T22:39:02Z

@mbolivar

I think that neither of these types of information is really a ControllerAndData due to the additional "child -> parent" information.

I am currently thinking it would make more sense to introduce two new types:
1. `NexusMapEntry`: for the table entries contained in a `<specifier>-map` as documented in DTSpec section 2.5.1. This would contain attributes named `child_specifier`, `parent`, `parent_specifier`.

2. `InterruptMapEntry`: similar, except for the `interrupt-map` property table entries in DTSpec 2.4.3.
This would match the same pattern we have done with the Range type used to represent elements of the standard ranges property. Experience has shown from that case that picking type names and designing APIs to match the specification leads to future-proof work.

What do you think about this idea?

This was possible with the current structure, so I did so, but
as mentioned in the comments, I think it would be clearer to define a new type.

There may be some room for debate as to whether to use NexusMapEntry/InterruptMapEntry or use MapEntry to make the interrupt property optional as needed for.

I'll move forward with it, but I might not have enough time this week.

mbolivar · 2025-10-06T23:46:00Z

There may be some room for debate as to whether to use NexusMapEntry/InterruptMapEntry or use MapEntry to make the interrupt property optional as needed for.

I agree there's room for debate. We could also use class inheritance to capture the specialization, perhaps. Thanks and looking forward to your next update.

soburi force-pushed the dt_map_properties branch 10 times, most recently from 32aac28 to 6f69749 Compare March 26, 2025 05:03

soburi changed the title ~~include: devicetree: Generate defines for map-related properties~~ include: devicetree: Generate defines for map property Mar 26, 2025

soburi force-pushed the dt_map_properties branch from 6f69749 to 812c000 Compare March 26, 2025 05:16

soburi marked this pull request as ready for review March 26, 2025 06:02

github-actions bot added area: Testsuite Testsuite area: ADC Analog-to-Digital Converter (ADC) area: Base OS Base OS Library (lib/os) area: GPIO area: Interrupt Controller area: Devicetree area: PCI Peripheral Component Interconnect labels Mar 26, 2025

github-actions bot requested review from aaronemassey, anangl, asemjonovs, dcpleung, decsny, galak, henrikbrixandersen, mnkp, nashif and peter-mitsis March 26, 2025 06:03

rruuaanng reviewed Jul 17, 2025

View reviewed changes

soburi force-pushed the dt_map_properties branch from cc1caeb to 0123d66 Compare August 2, 2025 05:09

soburi requested a review from rruuaanng August 2, 2025 06:00

rruuaanng reviewed Sep 11, 2025

View reviewed changes

soburi force-pushed the dt_map_properties branch 2 times, most recently from 5dc90ba to 6ace2ed Compare September 20, 2025 04:55

soburi requested a review from rruuaanng September 20, 2025 05:42

soburi mentioned this pull request Sep 25, 2025

zephyrCommon: Changing pin numbering rules zephyrproject-rtos/gsoc-2022-arduino-core#134

Open

mbolivar requested changes Oct 3, 2025

View reviewed changes

soburi added 2 commits October 4, 2025 19:45

sys: util: Add GET_ARGS_FIRST_N to get the first N of the args

cc63b68

Gets the specified number of elements from the beginning of the list. This is the symmetircal operation as `GET_ARGS_LESS_N`. Signed-off-by: TOKITA Hiroshi <[email protected]>

tests: unit: util: Add test for GET_ARGS_FIRST_N

0b91dd6

Add `GET_ARGS_FIRST_N` tests. Signed-off-by: TOKITA Hiroshi <[email protected]>

soburi force-pushed the dt_map_properties branch from 6ace2ed to 7f80024 Compare October 4, 2025 12:19

zephyrbot requested review from mbolivar and tejlmand October 4, 2025 12:21

mbolivar requested changes Oct 4, 2025

View reviewed changes

zephyrbot requested a review from mbolivar October 5, 2025 10:34

dts: bindings: Change *-map-mask and *-map-pass-thru as array type.

7bf09d5

They must be `array` type to match the existing `interrupt-map-mask` and `interrupt-map-pass-thru`. And also, updating descriptions. Signed-off-by: TOKITA Hiroshi <[email protected]>

soburi force-pushed the dt_map_properties branch from f8a1ce3 to 21c854e Compare October 5, 2025 10:39

soburi added 4 commits October 5, 2025 20:52

scripts: dts: devicetree: tests: Add a map property test

ab2d888

Add a map property test for `test_edtlib.py`. Signed-off-by: TOKITA Hiroshi <[email protected]>

tests: lib: devicetree: api: Add map property tests

b4a6f6a

Add tests for map property related macros. Add `native_sim/native/64` to allowed platforms. Signed-off-by: TOKITA Hiroshi <[email protected]>

soburi force-pushed the dt_map_properties branch from 21c854e to b4a6f6a Compare October 5, 2025 11:52

mbolivar reviewed Oct 5, 2025

View reviewed changes

doc/build/dts/macros.bnf Show resolved Hide resolved

		out_dt_define(macro, val)


		def write_maps(node: edtlib.Node) -> None:

include: devicetree: Generate defines for map property #87595

Are you sure you want to change the base?

include: devicetree: Generate defines for map property #87595

Uh oh!

Conversation

soburi commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Aug 2, 2025

Quality Gate passed

Uh oh!

soburi commented Sep 10, 2025

Uh oh!

rruuaanng left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

soburi Sep 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbolivar left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

soburi Oct 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mbolivar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sonarqubecloud bot commented Oct 5, 2025

Quality Gate passed

Uh oh!

mbolivar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mbolivar commented Oct 5, 2025

Uh oh!

soburi commented Oct 6, 2025

Uh oh!

mbolivar commented Oct 6, 2025

Uh oh!

Uh oh!

soburi commented Mar 24, 2025 •

edited

Loading

soburi Sep 20, 2025 •

edited

Loading

soburi Oct 4, 2025 •

edited

Loading

mbolivar left a comment •

edited

Loading